274 research outputs found
NasHD: Efficient ViT Architecture Performance Ranking using Hyperdimensional Computing
Neural Architecture Search (NAS) is an automated architecture engineering
method for deep learning design automation, which serves as an alternative to
the manual and error-prone process of model development, selection, evaluation
and performance estimation. However, one major obstacle of NAS is the extremely
demanding computation resource requirements and time-consuming iterations
particularly when the dataset scales. In this paper, targeting at the emerging
vision transformer (ViT), we present NasHD, a hyperdimensional computing based
supervised learning model to rank the performance given the architectures and
configurations. Different from other learning based methods, NasHD is faster
thanks to the high parallel processing of HDC architecture. We also evaluated
two HDC encoding schemes: Gram-based and Record-based of NasHD on their
performance and efficiency. On the VIMER-UFO benchmark dataset of 8
applications from a diverse range of domains, NasHD Record can rank the
performance of nearly 100K vision transformer models with about 1 minute while
still achieving comparable results with sophisticated models
PyGFI: Analyzing and Enhancing Robustness of Graph Neural Networks Against Hardware Errors
Graph neural networks (GNNs) have recently emerged as a promising learning
paradigm in learning graph-structured data and have demonstrated wide success
across various domains such as recommendation systems, social networks, and
electronic design automation (EDA). Like other deep learning (DL) methods, GNNs
are being deployed in sophisticated modern hardware systems, as well as
dedicated accelerators. However, despite the popularity of GNNs and the recent
efforts of bringing GNNs to hardware, the fault tolerance and resilience of
GNNs have generally been overlooked. Inspired by the inherent algorithmic
resilience of DL methods, this paper conducts, for the first time, a
large-scale and empirical study of GNN resilience, aiming to understand the
relationship between hardware faults and GNN accuracy. By developing a
customized fault injection tool on top of PyTorch, we perform extensive fault
injection experiments on various GNN models and application datasets. We
observe that the error resilience of GNN models varies by orders of magnitude
with respect to different models and application datasets. Further, we explore
a low-cost error mitigation mechanism for GNN to enhance its resilience. This
GNN resilience study aims to open up new directions and opportunities for
future GNN accelerator design and architectural optimization
Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applications
The challenging deployment of compute-intensive applications from domains
such Artificial Intelligence (AI) and Digital Signal Processing (DSP), forces
the community of computing systems to explore new design approaches.
Approximate Computing appears as an emerging solution, allowing to tune the
quality of results in the design of a system in order to improve the energy
efficiency and/or performance. This radical paradigm shift has attracted
interest from both academia and industry, resulting in significant research on
approximation techniques and methodologies at different design layers (from
system down to integrated circuits). Motivated by the wide appeal of
Approximate Computing over the last 10 years, we conduct a two-part survey to
cover key aspects (e.g., terminology and applications) and review the
state-of-the art approximation techniques from all layers of the traditional
computing stack. In Part II of our survey, we classify and present the
technical details of application-specific and architectural approximation
techniques, which both target the design of resource-efficient
processors/accelerators & systems. Moreover, we present a detailed analysis of
the application spectrum of Approximate Computing and discuss open challenges
and future directions.Comment: Under Review at ACM Computing Survey
Approximate Computing Survey, Part I: Terminology and Software & Hardware Approximation Techniques
The rapid growth of demanding applications in domains applying multimedia
processing and machine learning has marked a new era for edge and cloud
computing. These applications involve massive data and compute-intensive tasks,
and thus, typical computing paradigms in embedded systems and data centers are
stressed to meet the worldwide demand for high performance. Concurrently, the
landscape of the semiconductor field in the last 15 years has constituted power
as a first-class design concern. As a result, the community of computing
systems is forced to find alternative design approaches to facilitate
high-performance and/or power-efficient computing. Among the examined
solutions, Approximate Computing has attracted an ever-increasing interest,
with research works applying approximations across the entire traditional
computing stack, i.e., at software, hardware, and architectural levels. Over
the last decade, there is a plethora of approximation techniques in software
(programs, frameworks, compilers, runtimes, languages), hardware (circuits,
accelerators), and architectures (processors, memories). The current article is
Part I of our comprehensive survey on Approximate Computing, and it reviews its
motivation, terminology and principles, as well it classifies and presents the
technical details of the state-of-the-art software and hardware approximation
techniques.Comment: Under Review at ACM Computing Survey
- …